A Spatial-Temporal technique of Viseme Extraction: Application in Speech Recognition

Speech recognition is a basic component in several research projects nowadays. However, to understand a speech, hearing is not enough, it is sometimes necessary to see it. Indeed perspective studies proved that visual information brought by the interlocutor’s face in a degraded communication condition, contributed largely to the improvement of speechintelligibility. In fact several domains are concerned with the use of visual information such as e-learning, Human-Machine interaction, etc. This paper presents a method allowing to carry out a spatial-temporal tracking of some points of interest in the speaker’s face and to indicate the different configuration of the mouth through visemes. Later on these visemes will be associated to relatively precise physical measures like the spreading of the lips and mouth height, in order to establish a correlation between the phoneme and the viseme. The results of our experiment show that we can describe the whole French phonemes by the visemes. This work is subscribed among the CMCU Project undertaken in LIRIS Laboratory, CNRS, Ecole Centrale de Lyon, France, and in collaboration with Pr. Liming CHEN director of the laboratory and Mr. Mohsen ARDABILIAN FARD Assistant Professor in Ecole Centrale de Lyon, France .